Technological advances in super-resolution microscopy to study cellular processes.

Since its initial demonstration in 2000, far-field super-resolution light microscopy has undergone tremendous technological developments. In parallel, these developments have opened a new window into visualizing the inner life of cells at unprecedented levels of detail. Here, we review the technical details behind the most common implementations of super-resolution microscopy and highlight some of the recent, promising advances in this field.

What's new and what's next in diffusion MRI preprocessing.

Diffusion MRI (dMRI) provides invaluable information for the study of tissue microstructure and brain connectivity, but suffers from a range of imaging artifacts that greatly challenge the analysis of results and their interpretability if not appropriately accounted for. This review will cover dMRI artifacts and preprocessing steps, some of which have not typically been considered in existing pipelines or reviews, or have only gained attention in recent years: brain/skull extraction, B-matrix incompatibilities w.r.t the imaging data, signal drift, Gibbs ringing, noise distribution bias, denoising, between- and within-volumes motion, eddy currents, outliers, susceptibility distortions, EPI Nyquist ghosts, gradient deviations, B1 bias fields, and spatial normalization. The focus will be on "what's new" since the notable advances prior to and brought by the Human Connectome Project (HCP), as presented in the predecessing issue on "Mapping the Connectome" in 2013. In addition to the development of novel strategies for dMRI preprocessing, exciting progress has been made in the availability of open source tools and reproducible pipelines, databases and simulation tools for the evaluation of preprocessing steps, and automated quality control frameworks, amongst others. Finally, this review will consider practical considerations and our view on "what's next" in dMRI preprocessing.

Breast Cancer Segmentation Methods: Current Status and Future Potentials.

Early breast cancer detection is one of the most important issues that need to be addressed worldwide as it can help increase the survival rate of patients. Mammograms have been used to detect breast cancer in the early stages; if detected in the early stages, it can drastically reduce treatment costs. The detection of tumours in the breast depends on segmentation techniques. Segmentation plays a significant role in image analysis and includes detection, feature extraction, classification, and treatment. Segmentation helps physicians quantify the volume of tissue in the breast for treatment planning. In this work, we have grouped segmentation methods into three groups: classical segmentation that includes region-, threshold-, and edge-based segmentation; machine learning segmentation; and supervised and unsupervised and deep learning segmentation. The findings of our study revealed that region-based segmentation is frequently used for classical methods, and the most frequently used techniques are region growing. Further, a median filter is a robust tool for removing noise. Moreover, the MIAS database is frequently used in classical segmentation methods. Meanwhile, in machine learning segmentation, unsupervised machine learning methods are more frequently used, and U-Net is frequently used for mammogram image segmentation because it does not require many annotated images compared with other deep learning models. Furthermore, reviewed papers revealed that it is possible to train a deep learning model without performing any preprocessing or postprocessing and also showed that the U-Net model is frequently used for mammogram segmentation. The U-Net model is frequently used because it does not require many annotated images and also because of the presence of high-performance GPU computing, which makes it easy to train networks with more layers. Additionally, we identified mammograms and utilised widely used databases, wherein 3 and 28 are public and private databases, respectively.

Accelerating quantitative susceptibility and R2* mapping using incoherent undersampling and deep neural network reconstruction.

Quantitative susceptibility mapping (QSM) and R2* mapping are MRI post-processing methods that quantify tissue magnetic susceptibility and transverse relaxation rate distributions. However, QSM and R2* acquisitions are relatively slow, even with parallel imaging. Incoherent undersampling and compressed sensing reconstruction techniques have been used to accelerate traditional magnitude-based MRI acquisitions; however, most do not recover the full phase signal, as required by QSM, due to its non-convex nature. In this study, a learning-based Deep Complex Residual Network (DCRNet) is proposed to recover both the magnitude and phase images from incoherently undersampled data, enabling high acceleration of QSM and R2* acquisition. Magnitude, phase, R2*, and QSM results from DCRNet were compared with two iterative and one deep learning methods on retrospectively undersampled acquisitions from six healthy volunteers, one intracranial hemorrhage and one multiple sclerosis patients, as well as one prospectively undersampled healthy subject using a 7T scanner. Peak signal to noise ratio (PSNR), structural similarity (SSIM), root-mean-squared error (RMSE), and region-of-interest susceptibility and R2* measurements are reported for numerical comparisons. The proposed DCRNet method substantially reduced artifacts and blurring compared to the other methods and resulted in the highest PSNR, SSIM, and RMSE on the magnitude, R2*, local field, and susceptibility maps. Compared to two iterative and one deep learning methods, the DCRNet method demonstrated a 3.2% to 9.1% accuracy improvement in deep grey matter susceptibility when accelerated by a factor of four. The DCRNet also dramatically shortened the reconstruction time of single 2D brain images from 36-140 seconds using conventional approaches to only 15-70 milliseconds.

Fixel-based Analysis of Diffusion MRI: Methods, Applications, Challenges and Opportunities.

Diffusion MRI has provided the neuroimaging community with a powerful tool to acquire in-vivo data sensitive to microstructural features of white matter, up to 3 orders of magnitude smaller than typical voxel sizes. The key to extracting such valuable information lies in complex modelling techniques, which form the link between the rich diffusion MRI data and various metrics related to the microstructural organization. Over time, increasingly advanced techniques have been developed, up to the point where some diffusion MRI models can now provide access to properties specific to individual fibre populations in each voxel in the presence of multiple "crossing" fibre pathways. While highly valuable, such fibre-specific information poses unique challenges for typical image processing pipelines and statistical analysis. In this work, we review the "Fixel-Based Analysis" (FBA) framework, which implements bespoke solutions to this end. It has recently seen a stark increase in adoption for studies of both typical (healthy) populations as well as a wide range of clinical populations. We describe the main concepts related to Fixel-Based Analyses, as well as the methods and specific steps involved in a state-of-the-art FBA pipeline, with a focus on providing researchers with practical advice on how to interpret results. We also include an overview of the scope of all current FBA studies, categorized across a broad range of neuro-scientific domains, listing key design choices and summarizing their main results and conclusions. Finally, we critically discuss several aspects and challenges involved with the FBA framework, and outline some directions and future opportunities.

AIDeveloper: Deep Learning Image Classification in Life Science and Beyond.

Artificial intelligence (AI)-based image analysis has increased drastically in recent years. However, all applications use individual solutions, highly specialized for a particular task. Here, an easy-to-use, adaptable, and open source software, called AIDeveloper (AID) to train neural nets (NN) for image classification without the need for programming is presented. AID provides a variety of NN-architectures, allowing to apply trained models on new data, obtain performance metrics, and export final models to different formats. AID is benchmarked on large image datasets (CIFAR-10 and Fashion-MNIST). Furthermore, models are trained to distinguish areas of differentiated stem cells in images of cell culture. A conventional blood cell count and a blood count obtained using an NN are compared, trained on >1.2 million images, and demonstrated how AID can be used for label-free classification of B- and T-cells. All models are generated by non-programmers on generic computers, allowing for an interdisciplinary use.

Digital Pathology Initiatives and Experience of a Large Academic Institution During the Coronavirus Disease 2019 (COVID-19) Pandemic.

CONTEXT.­: Pathology practices have begun integrating digital pathology tools into their routine workflow. During 2020, the coronavirus disease 2019 (COVID-19), caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), emerged as a pandemic, causing a global health crisis that significantly affected the world population in several areas, including medical practice, and pathology was no exception. OBJECTIVE.­: To summarize our experience in implementing digital pathology for remote primary diagnosis, education, and research during this pandemic. DESIGN.­: We surveyed our pathologists (all subspecialized) and trainees to gather information about their use of digital pathology tools before and during the pandemic. Quality assurance and slide distribution data were also examined. RESULTS.­: During the pandemic, the widespread use of digital tools in our institution allowed a smooth transition of most clinical and academic activities into remote with no major disruptions. The number of pathologists using whole slide imaging (WSI) for primary diagnosis increased from 20 (62.5%) to 29 (90.6%) of a total of 32 pathologists, excluding renal pathology and hematopathology, during the pandemic. Furthermore, the number of pathologists exclusively using whole slide imaging for primary diagnosis also increased from 2 (6.3%) to 5 (15.6%) during the pandemic. In 35 (100%) survey responses from attending pathologists, 21 (60%) reported using whole slide imaging for remote primary diagnosis following the Centers for Medicare and Medicaid Services waiver. Of these 21 pathologists, 18 (86%) responded that if allowed, they will continue using whole slide imaging for remote primary diagnosis after the pandemic. CONCLUSIONS.­: The pandemic served as a catalyst to pathologists adopting a digital workflow into their daily practice and realizing the logistic and technical advantages of such tools.

Global evolution of research on pulmonary nodules: a bibliometric analysis.

Aim: To provide a historical and global picture of research concerning lung nodules, compare the contributions of major countries and explore research trends over the past 10 years. Methods: A bibliometric analysis of publications from Scopus (1970-2020) and Web of Science (2011-2020). Results: Publications about pulmonary nodules showed an enormous growth trend from 1970 to 2020. There is a high level of collaboration among the 20 most productive countries and regions, with the USA located at the center of the collaboration network. The keywords 'deep learning', 'artificial intelligence' and 'machine learning' are current hotspots. Conclusions: Abundant research has focused on pulmonary nodules. Deep learning is emerging as a promising tool for lung cancer diagnosis and management.

Radiomics MRI for lymph node status prediction in breast cancer patients: the state of art.

OBJECTIVES: To create a review of the existing literature on the radiomic approach in predicting the lymph node status of the axilla in breast cancer (BC). MATERIALS AND METHODS: Two reviewers conducted the literature search on MEDLINE databases independently. Ten articles on the prediction of sentinel lymph node metastasis in breast cancer with a radiomic approach were selected. The study characteristics and results were reported. The quality of the methodology was evaluated according to the Radiomics Quality Score (RQS). RESULTS: All studies were retrospective in design and published between 2017 and 2020. The majority of studies used DCE-MRI sequences and two investigated only pre-contrast images. The sample size was lower than 200 patients for 7 studies. The pre-processing used software, feature extraction and selection methods and classifier development are heterogeneous and a standardization of results is not yet possible. The average RQS score was 11.1 (maximum possible value = 36). The criteria with the lowest scores were the type of study, validation, comparison with a gold standard, potential clinical utility, cost-effective analysis and open science data. CONCLUSION: The field of radiomics is a diagnostic approach of relative recent development. The results in predicting axillary lymph node status are encouraging, but there are still weaknesses in the quality of studies that may limit the reproducibility of the results.

Changing the face of neuroimaging research: Comparing a new MRI de-facing technique with popular alternatives.

Recent advances in automated face recognition algorithms have increased the risk that de-identified research MRI scans may be re-identifiable by matching them to identified photographs using face recognition. A variety of software exist to de-face (remove faces from) MRI, but their ability to prevent face recognition has never been measured and their image modifications can alter automated brain measurements. In this study, we compared three popular de-facing techniques and introduce our mri_reface technique designed to minimize effects on brain measurements by replacing the face with a population average, rather than removing it. For each technique, we measured 1) how well it prevented automated face recognition (i.e. effects on exceptionally-motivated individuals) and 2) how it altered brain measurements from SPM12, FreeSurfer, and FSL (i.e. effects on the average user of de-identified data). Before de-facing, 97% of scans from a sample of 157 volunteers were correctly matched to photographs using automated face recognition. After de-facing with popular software, 28-38% of scans still retained enough data for successful automated face matching. Our proposed mri_reface had similar performance with the best existing method (fsl_deface) at preventing face recognition (28-30%) and it had the smallest effects on brain measurements in more pipelines than any other, but these differences were modest.

Virtual brain grafting: Enabling whole brain parcellation in the presence of large lesions.

Brain atlases and templates are at the heart of neuroimaging analyses, for which they facilitate multimodal registration, enable group comparisons and provide anatomical reference. However, as atlas-based approaches rely on correspondence mapping between images they perform poorly in the presence of structural pathology. Whilst several strategies exist to overcome this problem, their performance is often dependent on the type, size and homogeneity of any lesions present. We therefore propose a new solution, referred to as Virtual Brain Grafting (VBG), which is a fully-automated, open-source workflow to reliably parcellate magnetic resonance imaging (MRI) datasets in the presence of a broad spectrum of focal brain pathologies, including large, bilateral, intra- and extra-axial, heterogeneous lesions with and without mass effect. The core of the VBG approach is the generation of a lesion-free T1-weighted image, which enables further image processing operations that would otherwise fail. Here we validated our solution based on Freesurfer recon-all parcellation in a group of 10 patients with heterogeneous gliomatous lesions, and a realistic synthetic cohort of glioma patients (n = 100) derived from healthy control data and patient data. We demonstrate that VBG outperforms a non-VBG approach assessed qualitatively by expert neuroradiologists and Mann-Whitney U tests to compare corresponding parcellations (real patients U(6,6) = 33, z = 2.738, P < .010, synthetic-patients U(48,48) = 2076, z = 7.336, P < .001). Results were also quantitatively evaluated by comparing mean dice scores from the synthetic-patients using one-way ANOVA (unilateral VBG = 0.894, bilateral VBG = 0.903, and non-VBG = 0.617, P < .001). Additionally, we used linear regression to show the influence of lesion volume, lesion overlap with, and distance from the Freesurfer volumes of interest, on labeling accuracy. VBG may benefit the neuroimaging community by enabling automated state-of-the-art MRI analyses in clinical populations using methods such as FreeSurfer, CAT12, SPM, Connectome Workbench, as well as structural and functional connectomics. To fully maximize its availability, VBG is provided as open software under a Mozilla 2.0 license (https://github.com/KUL-Radneuron/KUL_VBG).

The nonhuman primate neuroimaging and neuroanatomy project.

Multi-modal neuroimaging projects such as the Human Connectome Project (HCP) and UK Biobank are advancing our understanding of human brain architecture, function, connectivity, and their variability across individuals using high-quality non-invasive data from many subjects. Such efforts depend upon the accuracy of non-invasive brain imaging measures. However, 'ground truth' validation of connectivity using invasive tracers is not feasible in humans. Studies using nonhuman primates (NHPs) enable comparisons between invasive and non-invasive measures, including exploration of how "functional connectivity" from fMRI and "tractographic connectivity" from diffusion MRI compare with long-distance connections measured using tract tracing. Our NonHuman Primate Neuroimaging & Neuroanatomy Project (NHP_NNP) is an international effort (6 laboratories in 5 countries) to: (i) acquire and analyze high-quality multi-modal brain imaging data of macaque and marmoset monkeys using protocols and methods adapted from the HCP; (ii) acquire quantitative invasive tract-tracing data for cortical and subcortical projections to cortical areas; and (iii) map the distributions of different brain cell types with immunocytochemical stains to better define brain areal boundaries. We are acquiring high-resolution structural, functional, and diffusion MRI data together with behavioral measures from over 100 individual macaques and marmosets in order to generate non-invasive measures of brain architecture such as myelin and cortical thickness maps, as well as functional and diffusion tractography-based connectomes. We are using classical and next-generation anatomical tracers to generate quantitative connectivity maps based on brain-wide counting of labeled cortical and subcortical neurons, providing ground truth measures of connectivity. Advanced statistical modeling techniques address the consistency of both kinds of data across individuals, allowing comparison of tracer-based and non-invasive MRI-based connectivity measures. We aim to develop improved cortical and subcortical areal atlases by combining histological and imaging methods. Finally, we are collecting genetic and sociality-associated behavioral data in all animals in an effort to understand how genetic variation shapes the connectome and behavior.

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI.

Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning (DL). Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example, the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the DL is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide "obviously" interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that: 1) clinicians and practitioners can subsequently approach these methods with caution; 2) insight into interpretability will be born with more considerations for medical practices; and 3) initiatives to push forward data-based, mathematically grounded, and technically grounded medical education are encouraged.

Automation of data analysis in molecular cancer imaging and its potential impact on future clinical practice.

Digitalization, especially the use of machine learning and computational intelligence, is considered to dramatically shape medical procedures in the near future. In the field of cancer diagnostics, radiomics, the extraction of multiple quantitative image features and their clustered analysis, is gaining increasing attention to obtain more detailed, reproducible, and meaningful information about the disease entity, its prognosis and the ideal therapeutic option. In this context, automation of diagnostic procedures can improve the entire pipeline, which comprises patient registration, planning and performing an imaging examination at the scanner, image reconstruction, image analysis, and feeding the diagnostic information from various sources into decision support systems. With a focus on cancer diagnostics, this review article reports and discusses how computer-assistance can be integrated into diagnostic procedures and which benefits and challenges arise from it. Besides a strong view on classical imaging modalities like x-ray, CT, MRI, ultrasound, PET, SPECT and hybrid imaging devices thereof, it is outlined how imaging data can be combined with data deriving from patient anamnesis, clinical chemistry, pathology, and different omics. In this context, the article also discusses IT infrastructures that are required to realize this integration in the clinical routine. Although there are still many challenges to comprehensively implement automated and integrated data analysis in molecular cancer imaging, the authors conclude that we are entering a new era of medical diagnostics and precision medicine.

Interval-Based Least Squares for Uncertainty-Aware Learning in Human-Centric Multimedia Systems.

Machine learning (ML) methods are popular in several application areas of multimedia signal processing. However, most existing solutions in the said area, including the popular least squares, rely on penalizing predictions that deviate from the target ground-truth values. In other words, uncertainty in the ground-truth data is simply ignored. As a result, optimization and validation overemphasize a single-target value when, in fact, human subjects themselves did not unanimously agree to it. This leads to an unreasonable scenario where the trained model is not allowed the benefit of the doubt in terms of prediction accuracy. The problem becomes even more significant in the context of more recent human-centric and immersive multimedia systems where user feedback and interaction are influenced by higher degrees of freedom (leading to higher levels of uncertainty in the ground truth). To ameliorate this drawback, we propose an uncertainty aware loss function (referred to as [Formula: see text]) that explicitly accounts for data uncertainty and is useful for both optimization (training) and validation. As examples, we demonstrate the utility of the proposed method for blind estimation of perceptual quality of audiovisual signals, panoramic images, and images affected by camera-induced distortions. The experimental results support the theoretical ideas in terms of reducing prediction errors. The proposed method is also relevant in the context of more recent paradigms, such as crowdsourcing, where larger uncertainty in ground truth is expected.

MR Elastography of the Breast: Evolution of Technique, Case Examples, and Future Directions.

Recognizing that breast cancers present as firm, stiff lesions, the foundation of breast magnetic resonance elastography (MRE) is to combine tissue stiffness parameters with sensitive breast MR contrast-enhanced imaging. Breast MRE is a non-ionizing, cross-sectional MR imaging technique that provides for quantitative viscoelastic properties, including tissue stiffness, elasticity, and viscosity, of breast tissues. Currently, the technique continues to evolve as research surrounding the use of MRE in breast tissue is still developing. In the setting of a newly diagnosed cancer, associated desmoplasia, stiffening of the surrounding stroma, and necrosis are known to be prognostic factors that can add diagnostic information to patient treatment algorithms. In fact, mechanical properties of the tissue might also influence breast cancer risk. For these reasons, exploration of breast MRE has great clinical value. In this review, we will: (1) address the evolution of the various MRE techniques; (2) provide a brief overview of the current clinical studies in breast MRE with interspersed case examples; and (3) suggest directions for future research.

Radiomics in neuro-oncology: Basics, workflow, and applications.

Over the last years, the amount, variety, and complexity of neuroimaging data acquired in patients with brain tumors for routine clinical purposes and the resulting number of imaging parameters have substantially increased. Consequently, a timely and cost-effective evaluation of imaging data is hardly feasible without the support of methods from the field of artificial intelligence (AI). AI can facilitate and shorten various time-consuming steps in the image processing workflow, e.g., tumor segmentation, thereby optimizing productivity. Besides, the automated and computer-based analysis of imaging data may help to increase data comparability as it is independent of the experience level of the evaluating clinician. Importantly, AI offers the potential to extract new features from the routinely acquired neuroimages of brain tumor patients. In combination with patient data such as survival, molecular markers, or genomics, mathematical models can be generated that allow, for example, the prediction of treatment response or prognosis, as well as the noninvasive assessment of molecular markers. The subdiscipline of AI dealing with the computation, identification, and extraction of image features, as well as the generation of prognostic or predictive mathematical models, is termed radiomics. This review article summarizes the basics, the current workflow, and methods used in radiomics with a focus on feature-based radiomics in neuro-oncology and provides selected examples of its clinical application.

Laplacian Pyramid Neural Network for Dense Continuous-Value Regression for Complex Scenes.

Many computer vision tasks, such as monocular depth estimation and height estimation from a satellite orthophoto, have a common underlying goal, which is regression of dense continuous values for the pixels given a single image. We define them as dense continuous-value regression (DCR) tasks. Recent approaches based on deep convolutional neural networks significantly improve the performance of DCR tasks, particularly on pixelwise regression accuracy. However, it still remains challenging to simultaneously preserve the global structure and fine object details in complex scenes. In this article, we take advantage of the efficiency of Laplacian pyramid on representing multiscale contents to reconstruct high-quality signals for complex scenes. We design a Laplacian pyramid neural network (LAPNet), which consists of a Laplacian pyramid decoder (LPD) for signal reconstruction and an adaptive dense feature fusion (ADFF) module to fuse features from the input image. More specifically, we build an LPD to effectively express both global and local scene structures. In our LPD, the upper and lower levels, respectively, represent scene layouts and shape details. We introduce a residual refinement module to progressively complement high-frequency details for signal prediction at each level. To recover the signals at each individual level in the pyramid, an ADFF module is proposed to adaptively fuse multiscale image features for accurate prediction. We conduct comprehensive experiments to evaluate a number of variants of our model on three important DCR tasks, i.e., monocular depth estimation, single-image height estimation, and density map estimation for crowd counting. Experiments demonstrate that our method achieves new state-of-the-art performance in both qualitative and quantitative evaluation on the NYU-D V2 and KITTI for monocular depth estimation, the challenging Urban Semantic 3D (US3D) for satellite height estimation, and four challenging benchmarks for crowd counting. These results demonstrate that the proposed LAPNet is a universal and effective architecture for DCR problems.